Method and system of adaptable exposure control and light projection for cameras

ABSTRACT

Techniques related to a method and system of adaptable exposure control and light projection for cameras are described herein.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. provisional patentapplication No. 62/170,126, filed Jun. 2, 2015, which is incorporatedherein for all purposes.

BACKGROUND

Many electronic imaging devices, such as smartphones or tablets with oneor more cameras actively illuminate the scene to facilitate extractionof non-traditional information from the scene. One example is depthmeasurement to objects in a scene, or measurement of the dimensions ofan object in the scene, during image capture of the scene. One suchsystem on these devices is an active stereoscopic system that can beused to generate images of objects in 3D space where each pixel isdefined as having depth values in addition to the Red-Green-Blue (RGB)values typical in a 2D system. Active stereo systems illuminate thescene to provide detectable features to be used for depth measurementwhich enhances the cameras performance with respect to objectscontaining few naturally visible features for the stereo system to matchand triangulate. Other examples of systems incorporating activeillumination include structured light, which illuminates the scene witha specific pattern and uses that pattern to triangulate the individuallyrecognized projected features; and coded light which projects a timevarying pattern and analyzes distortions in the pattern to infer depth.Non-depth applications include iris scanning and face recognition, bothof which use invisible illumination to reduce the effects of naturallyoccurring shadows.

In one specific example of active stereo depth cameras, two or morecameras may be used to measure depths of an object in a scene by usingtriangulation. This involves measuring the angles from a common objectin the scene from two or more separate cameras or sensors. When theorientation and distance between the two or more cameras is known, thedepth of the common object can be calculated.

The active stereo example resolves the case of there being insufficientobject points in a scene to determine a triangulation, such as an imageof a plain white wall. To accommodate these scenes, the active stereocamera includes a projector, such as an infra-red laser projector, toproject an intensity pattern onto the scene. The stereo cameras, or IRsensors, may then use the intensity pattern to measure the depth of theobject that the intensity pattern is projected upon. Difficulties oftenoccur, however, when the scene is outside in sunlight. The sunlight cansaturate the light entering the imaging device thereby resulting in alow signal-to-noise ratio (SNR), and this may even entirely wash out theprojected intensity pattern.

In other examples, such as structured light or coded light, any externalsources of illumination in the wavelength being used can interfere withthe operation of the system including other systems employing similartechnology. The sensor often cannot discriminate between light projectedby the systems camera and light projected by a second camera, or othersystem using the same type of illumination. Therefore, light from asecond system, or second camera on the same system, can distort theperceived scene and corrupt the depth measurement, or face recognitionor iris recognition or any other application where the system requirescontrol of the scene illumination for optimum performance.

DESCRIPTION OF THE FIGURES

The material described herein is illustrated by way of example and notby way of limitation in the accompanying figures. For simplicity andclarity of illustration, elements illustrated in the figures are notnecessarily drawn to scale. For example, the dimensions of some elementsmay be exaggerated relative to other elements for clarity. Further,where considered appropriate, reference labels have been repeated amongthe figures to indicate corresponding or analogous elements. In thefigures:

FIG. 1 is a schematic diagram to assist with explaining assistedstereoscopic depth measurement.

FIG. 1A is an example imaging device with depth measurement andspatially varying intensity pattern projection and shown in operation inaccordance with the implementations herein;

FIG. 1B is a another schematic diagram of an example imaging device withdepth measurement and spatially varying intensity pattern projection inaccordance with the implementations herein;

FIG. 2 is an example projected spatially varying intensity pattern usedfor the implementations herein;

FIG. 3 is a flow chart of a method of exposure control and patternprojection in accordance with the implementations herein;

FIG. 4 is a schematic diagram of a depth measurement system to providedepth measurements on an imaging device;

FIG. 5 is a detailed flow chart of a method of exposure control andpattern projection in accordance with the implementations herein;

FIG. 6 is a solar radiation graph;

FIG. 7 is a chart of thermal limitations for continuous wave laseroperation;

FIG. 8 is a chart of eye safety limitations for pulsed laser operations;

FIG. 9 is a chart of average power limitations for modulated laseroperation;

FIG. 10 is a graph of conventional projection times used withconventional exposure times;

FIG. 11 is a graph of possible enhanced projection times used withconventional exposure times;

FIG. 12 is a graph of projection and exposure times applyingimplementations herein;

FIG. 13 is an illustrative diagram of an example image processing systemaccording to the implementations herein;

FIG. 14 is an illustrative diagram of an example system; and

FIG. 15 is an illustrative diagram of an example system, all arranged inaccordance with at least some implementations of the present disclosure.

DETAILED DESCRIPTION

One or more implementations are now described with reference to theenclosed figures. While specific configurations and arrangements arediscussed, it should be understood that this is performed forillustrative purposes only. Persons skilled in the relevant art willrecognize that other configurations and arrangements may be employedwithout departing from the spirit and scope of the description. It willbe apparent to those skilled in the relevant art that techniques and/orarrangements described herein may also be employed in a variety of othersystems and applications other than what is described herein

While the following description sets forth various implementations thatmay be manifested in architectures such as system-on-a-chip (SoC)architectures for example, implementation of the techniques and/orarrangements described herein are not restricted to particulararchitectures and/or computing systems and may be implemented by anyarchitecture and/or computing system for similar purposes. For instance,various architectures employing, for example, multiple integratedcircuit (IC) chips and/or packages, and/or various computing devicesand/or consumer electronic (CE) devices such as imaging devices, digitalcameras, camera arrays, smart phones, webcams, video game panels orconsoles, set top boxes, tablets with multiple cameras, and so forth,may implement the techniques and/or arrangements described herein.Further, while the following description may set forth numerous specificdetails such as logic implementations, types and interrelationships ofsystem components, logic partitioning/integration choices, and so forth,claimed subject matter may be practiced without such specific details.In other instances, some material such as, for example, controlstructures and full software instruction sequences, may not be shown indetail in order not to obscure the material disclosed herein. Thematerial disclosed herein may be implemented in hardware, firmware,software, or any combination thereof.

The material disclosed herein may also be implemented as instructionsstored on a machine-readable medium, which may be read and executed byone or more processors. A machine-readable medium may include any mediumand/or mechanism for storing or transmitting information in a formreadable by a machine (for example, a computing device). For example, amachine-readable medium may include read-only memory (ROM); randomaccess memory (RAM); magnetic disk storage media; optical storage media;flash memory devices; electrical, optical, acoustical or other forms ofpropagated signals (e.g., carrier waves, infrared signals, digitalsignals, and so forth), and others. In another form, a non-transitoryarticle, such as a non-transitory computer readable medium, may be usedwith any of the examples mentioned above or other examples except thatit does not include a transitory signal per se. It does include thoseelements other than a signal per se that may hold data temporarily in a“transitory” fashion such as RAM and so forth.

References in the specification to “one implementation”, “animplementation”, “an example implementation”, and so forth, indicatethat the implementation described may include a particular feature,structure, or characteristic, but every implementation may notnecessarily include the particular feature, structure, orcharacteristic. Moreover, such phrases are not necessarily referring tothe same implementation. Further, when a particular feature, structure,or characteristic is described in connection with an implementation, itis submitted that it is within the knowledge of one skilled in the artto effect such feature, structure, or characteristic in connection withother implementations whether or not explicitly described herein.

Systems, articles, and methods to provide adaptable exposure control andlight projection for cameras is described below.

As mentioned above and while referring to FIG. 1, depth of an object ina scene being photographed or recorded may be determined with a 3Dcapable camera or imaging device with multiple cameras by determiningthe shift in image features as seen by two or more cameras separated bysome distance (called the baseline) using assisted 3D stereoscopic depthcameras. Specifically, a stereo depth measurement system 10 is shownwith two cameras or sensors on an xyz coordinate system, where thecenter of one sensor and origin of the optical axis is located at (0, 0,0) and the center of another sensor and optical axis origin is at (Tx,0, 0) where the two sensors are Tx apart from each and where Tx formsthe baseline. Two left and right images 12 and 14 are respectivelyformed by the two cameras or sensors. An object, or key point of anobject, 20 is at point (X, Y, Z). The 2D coordinates for the points 16and 18 and of the object 20 on the left and right images 12 and 14respectively can be calculated by triangulation. For purposes ofillustration it can be shown that image coordinate points 16 and 18 ofobject 20 for the left sensor can be calculated as:

x ₁=ƒ(X/Z)y ₁=ƒ(Y/Z)

where x₁ and y₁ merely refer to the (x, y) coordinates of the leftsensor (or left image). A similar calculation is performed for the rightsensor. The function f( ) is based on the optical properties of theoptical system and sensor arrangement. Since the cameras are on a noncoincident baseline, features on the left and right sensors will beoffset. This offset is referred to as disparity. By locating featureswith known feature matching algorithms on both sensors, the disparitycan be determined. The disparity in turn can be used with knownalgorithms to determine the distance from the object or key points ofthe object. Such a depth measurement system that is dependent on thedetection of features in a scene does not work well, however, on afeatureless object or scene being photographed such as a blank wallwhere there are no key feature points that are detectable by a camera'ssensors. One conventional solution to this is active illumination thatincludes projecting artificial features onto the scene instead. This maybe accomplished by using either an LED or laser to project a lightpattern onto the scene, typically in infra-red (IR) radiation and toprovide feature points that can be detected by IR sensors to betterensure that depth can be calculated for an otherwise featureless object.

In the outdoors, however, sunlight dominates the light and can causesolar interference that causes a low signal-to-noise ratio, and can evenwash out the projection. The sun emits light with a content thatincludes a strong 850 nm light, and most systems typically project theirpatterns at about the same wavelength. Projection of a pattern at 850 nmwas used because it has been found to be a good compromise among sensorresponse, laser efficiency and eye safety, except in bright sunlight.Thus, the system often did not work well unless it was indoors or atnight.

Thus, it is desirable to provide a very bright projector to betterensure the detectability of the pattern. Boosting the brightness of theprojector also may extend the working range from the camera. Thissolution, however, requires an increase in power that might not meet eyesafety standards or thermal limitations.

To avoid thermal problems, the exposure time and projection time couldbe reduced. For example, rather than having the shutter open for a fullframe period, the shutter may be closed for a portion of the frameperiod for a set amount of time. This, however, remains limited by themaximum eye safety power for frame rates below approximately 200 KHzwhereas current systems operate at a maximum frame rate of 60 Hz.

To resolve these issues then, the present method and system discloses acompromise solution that modulates the projection during at least aportion of the frame period and with a modulation frequency that meetthe requirements to apply an average eye safety threshold rather thanthe peak threshold. This permits a boost in power that is less likely tobe washed out by sunlight. The sunlight rejection can be furtherenhanced by switching the wavelength of the projector and sensor to 940nm because the atmosphere absorbs relatively more energy at the desiredwavelength of 940 nm resulting in less available sunlight at 940 nm atlow altitudes. Also, the exposure time at the sensors is modulated sothat the sensor collects light signals during a pulse and discardsenergy between the pulses. By synchronizing the exposure time to thepulses, sunlight rejection may be improved by 10 to 1000 times therebysubstantially increasing the pattern detection efficiency. While thesynchronization method at the new wavelength may be performed by avariety of different sensor and projector technologies, the use of aVCSEL laser projector coupled with sensors that have a quantum film isexpected to provide a substantial improvement in efficient sunlightrejection as described below.

Referring to FIG. 1A, an imaging device 100 is provided to measure the3D or depth of an object 104 in a scene being photographed or recordedin a video recording. The object may be a featureless object such as aplain white wall with no clear distinct features that can be detected byconventional RGB detection techniques. Here, a projector 110 isprovided, which may be an IR varying intensity pattern projector. Theprojector may be LED, laser, or VCSEL as described herein, and projectsa pattern 106 on the object 104 as shown by the solid lines fromprojector 108 to pattern 106. Such a pattern may be similar to pattern200 (FIG. 2), which is a projection of a varying light pattern at atarget IR, or near IR (NIR), wavelength such as about 940 nm, or about939 nm to about 941 nm. The projector is arranged to use higher powersettings and to emit the wavelengths needed to avoid sunlight washout,and that are pulsed in configurations that avoid exceeding eye safetyaverage thresholds for image frame periods as explained below.

The imaging device 100 also may have left and right IR sensors 108 and112 that detect the pattern 106 by the dashed lines from the pattern 106to the sensors 108 and 112. The pattern is sufficiently large so thatdetection of a feature by both sensors is considered to be the samepoint and is used for triangulation calculations to determine a depthdistance from the imaging device 100 to the object 104. The sensors alsoeach have an electronic shutter to control the light entering thesensors. The exposure can be controlled so that the exposure times aresynchronized with the pulses of the projector in order to minimizesunlight rejection. An RGB sensor 114 also may be provided and used todetermine the visual color and appearance of the object being measured.The details of the exposure and IR projection control are providedbelow.

Referring to FIG. 3, an example process 300 for adaptable exposurecontrol and light projection for cameras described herein is arranged inaccordance with at least some implementations of the present disclosure.In the illustrated implementation, process 300 may include one or moreoperations, functions or actions as illustrated by one or more ofoperations 302 and 306. By way of non-limiting example, process 300 willbe described herein with reference to example image processing system ofFIGS. 1A, 4, and 13 where relevant.

By one implementation of an image processing system, process 300 mayinclude “PROJECT LIGHT IN MULTIPLE PULSES IN A SINGLE FRAME PERIOD,WHEREIN THE FRAME PERIOD IS USED TO CAPTURE A SINGLE IMAGE OF A SCENE”302. As explained herein, such light may in the form of a pattern, suchas a geometrical pattern by one example, that may be projected onto socalled featureless objects that have few detectable points that may berecognized by a typical camera for depth measurement for example. Thus,the pattern projected onto an object provides the points that can bedetected by a camera sensor. The pattern may be undetectable by thehuman eye such in the near infra-red zone (IR or NIR) which is about 700nm to 1000 nm wavelengths. The wavelength for this method, however, iseven more specific as explained below.

In one example, the pulses collectively have a total duration that formsa proportion of the total frame period. The total frame period includesthe time the pulses are off before, after, and/or between the pulses.The proportion has a duration that results in an average power togenerate the pulses over the frame period that is below a maximum eyesafety threshold for the power of a single frame period. The pulses havea frequency greater than a certain threshold (such as greater than 200kHz by one possible example) so that the eye safety threshold iscomputed as an average power over the frame period rather than a peakthreshold, which enables a higher peak power in combination with awavelength with relatively less energy in sunlight. By one form, thewavelength is set at about 940 nm, or in a range of about 939 nm to 941nm. It also will be understood that the example eye safety threshold(200 kHz) may be very different depending on the laser configuration andtype, aperture, peak power profile, divergent angles, and other factors.It also will be understood that the average power of a frame period maynot be the only threshold that needs to be complied with for eye safety.

Process 300 also may include “PERFORM MULTIPLE EXPOSURE TIMES DURING THESINGLE FRAME PERIOD AND AT AT LEAST ONE SENSOR USED TO DETECT THE SCENE”304. Thus, the sensors are modulated shutter sensors that are able toperform multiple exposure times in a single frame period. As explainedbelow, this may be used to limit sunlight rejection (or domination) ofthe desired light capture from the projected pattern. The exposure timesmay be set to provide an open shutter during one or more projections (orpulses) of the pattern.

By this alternative, process 300 also may include “SYNCHRONIZE THEEXPOSURE TIMES TO THE PULSES SO THAT A SINGLE EXPOSURE TIME CORRESPONDSTO A SINGLE PULSE DURATION FOR MULTIPLE INDIVIDUAL PULSES DURING ASINGLE FRAME PERIOD” 306, and particularly, so that the sensor exposureis started at or near the start of an individual pulse and ended at ornear the end of an individual pulse for multiple pulses, and by oneexample, for each projector pulse during the frame period. The exposuretime is limited to the pulse duration to further limit rejection of thedesired light from the pattern by sunlight.

It also will be understood that any discussion of the pulses matching orcorresponding to exposure times (or vice versa) or that both occur atthe same time may refer to generally or substantially occurring at thesame time depending on what is needed or desired to block a sufficientamount of sunlight. Thus, the exposure times may be slightly different(such as starting earlier and/or ending later) to ensure no pulseduration is undesirably cut off from inclusion by the shutter forexample, and in units relative to the methods herein (such as measuredin ns).

Referring to FIG. 4, an image capture system 400 is arranged to performthe implementations herein. The image capture system 400 has an exposurecontrol unit 402 with a pattern synchronization unit 404. The exposurecontrol unit 402 sets the timing of the starting and stopping of theexposure on pattern sensors 412 and 418 via sensor controls 408 and 409.A pattern control unit 406 sets the timing and other parameters for aprojector driver 410 that operates a projector 416. An RGB sensor 414may be provided as well when the pattern sensors are not also RGBsensors. The RGB sensor 414 may be used to provide pixel color imagedata.

System 400 may have many different configurations in order to implementthe exposure control and pattern projection methods herein. Thus, thesystem 400 may be a single device (such as a single camera with multiplesensors) or multiple devices (such as multiple cameras), or may includeseparate components on a single device (such as a smartphone or tabletwith multiple cameras and a separate projector). By one example, thesensors 412, 418, sensor controls 408, 409, and exposure control unit402 may be considered to establish, or be part of, one or more cameras,while the pattern control unit 406, projector control 410, and patternprojector 416 may establish, or may be considered to be part of, aprojector (or laser or light source) separate from the cameras, but mayor may not be on the same device, such as on a smartphone or tablet.There also may be a pulse generator 422 that may be considered part ofthe pattern synchronization unit 404 and one or more cameras, and/or apulse generator 424 considered to be part of pattern control unit 406and/or a light source. Otherwise, a pulse generator 426 may be part ofeither, both, or neither the camera(s) and the projector, and may beprogrammable. The location of the pulse generator may depend upon whichcomponent(s) are considered the slave, and which are considered themaster as explained in greater detail below.

A microcontroller may provide one of the programmable pulse generatorsto vary the pulse width and frequency so that it is a fairly flexiblesystem. The microcontroller can be programmed with default values forthe pulse generator and the frequency. Alternatively, a communicationlink 427, such as i2c, may be used to change the pulse settings. For oneexample, a pre-defined sequences of pulses (such as a burst at thebeginning of the frame period), may be set at the pulse generator 424.These could be selected over the communication link 427. In this lattersituation, where the pulse generator at the laser sets the pulse rate,the laser is considered the master unit.

In a typical CMOS sensor the duration of the interval between resettingthe pixel and reading it is used to determine the exposure time. In theCMOS sensor utilized here, a transfer gate or other electronic method isused to determine the exposure time of the pixels and may beenabled/disabled multiple times during the exposure. This can be used toprovide the multiple exposures during a single frame period. In thissystem, either the exposure gate is controlled by an external pin for“slaving” the sensor to the IR projector light source as explainedherein, or an on-board programmable pulse generator is used to providean appropriate pulse to synchronize the light source. In conventionalshutter cameras, the pulse generator for the reset and read gatescontrols duration and the phase (time from the start of frame). In thepresent method, an additional parameter, pulse rate, would be used alsoas described below.

Specifically, these components provide the ability to synchronize theexposures to the pulsed light source during the frame in many differentways. The camera (sensor(s)) may be slaves to the projector, or theprojector may be a slave to the camera(s), or there may be a combinationof both, and by one form depending on which component has the pulsegenerator, in order to synchronize the timing of when the exposures takeplace.

When the camera (sensors) is a slave to the master light source, theslave camera(s) receive an input from the master light source, or inother words, the master light source provides a synchronization to theslave camera. In this case, the camera sensors accept electricalsynchronization signal(s) that would allow an external source to controlthe pattern of exposures of the camera's exposure control.

By this example of system 400, the pattern control unit 406 is themaster and has pattern settings which may be set via pulse generator 424or a communication link 427 thereto. The pulse settings may include thefrequency, duration (or pulse width), and non-uniform (or bunching)timing of the pulses when provided (which may be considered to beincluded in the frequency determination), and per frame period, as wellas the power. The target emission wavelength of the projection pulsesmay be fixed and set in compliance with sensor configurationlimitations, such as on an IR CMOS sensor (or IR RGB CMOS sensor)arranged to detect 940 nm light. The settings are provided to theprojector driver 410 to operate the projector 416 to emit the desiredpulse pattern through a lens 423 by controlling a laser 430.

The settings also may be provided to the pattern synchronization unit404 so that exposure time settings may be set to match the durations ofthe individual pulses. The exposure time settings are then provided tothe sensor control 408 or 409 to control the exposure on the sensor 412or 418 to perform the exposure timing according to the exposure settingsand detect the light from the projection through lenses 438 and 440.

Alternatively, when the light source (projector/laser) is a slave to themaster camera(s) or sensor(s), the master camera transmits an output tothe slave light source. In this case, the slave light source can betriggered with an external synchronization from the master camera, andthe camera may have the controlling pulse generator to communicate tothe laser to trigger the generation of a pulse. Specifically, for theexample of system 400, now the pattern synchronization unit 404 of theexposure control unit 402 may have the pulse generator 422 to sendelectronic signals to the pattern control unit 406 to set the pulseparameters at the projector driver 410 to control the projector 416 inaccordance with a pulse plan or pattern (or pulse rate). The patternsynchronization unit 404 also would set the exposure durations accordingto the pulse pattern, and provide the exposure parameter settings to thesensor control 408 and 409 to control the exposure on the sensor 412 or418 to perform the exposure timing according to the pulse pattern.

There are many options as to exactly what and how the camera may controlthe pulses, and what may be included in the pulse plan or pattern (orpulse rate). The camera output pulse/pulses could be a simple trigger toinitiate a pre-set pulse pattern (or pulse rate), or also could controlthe width of the pulse. If the pulse generator 422 resides in thecamera, there may be an interface, such as for communication link 427,where one could communicate with the camera to define the pulseparameters. The camera could also adjust the number of pulses tofacilitate exposure. The pulse generator in the camera could havefeatures that help to ensure safety and proper laser operation asdescribed herein. For example, it might have limits to the number ofpulses/frame or the pulse width or a combination of both. Also, when thecamera(s) is a master, the camera may generate signal(s) that indicatewhen an exposure was taking place, and the light source may acceptelectrical synchronization signals to provide light during the exposureperiod. It will be understood of course that the pattern control unit406 may make many of these settings when the projector is the mastercomponent.

By yet another option, the pulse generator 426 may be provided, and maybe considered separate from both the camera(s) and projector, such as aseparate microcontroller communicating with camera(s) and projector on asmartphone or tablet for example. The pulse generator 426 may performany of the tasks that could be performed by pulse generator 422 and 424.

The sensors 412 and 418 may be complimentary metal-oxide silicon (CMOS)sensors that work more like a traditional film in that the whole imageis “exposed” at the same time. The sensor may be pixelated so thatphotons (and in turn charge) are collected for each pixel (or block ofpixels) locations on the sensor surface and all the pixels are exposedin unison for a period of time as in global shutter operation, or arestaged such that the exposure time ends just prior to the readout ofeach row as in rolling shutter mode of operation. The camera here,however, can be integrated for short durations and triggered multipletimes during a single frame period. Triggered multiple times refers toperforming an exposure more than once during the frame time or period,and the frame period is the period of time where an exposure isaccumulated for a reading and capturing of a single image, before it isreset for the next frame.

The sensor may each include switches (transistors) that control when thecapacitor integration occurs (or in other words, exposure when thephotosensitive part is connected to a capacitor of the sensor) and whenit is read (the capacitor is connected to an analog-digital convertor(ADC)), and when it is reset (where the capacitor is set to the blacklevel). For shorter exposures during a single frame period (such asthose at an IR pulse as described herein), the photosensitive portion isconnected and disconnected to and from the capacitor without the readand reset operations, thereby collecting the charge during the multiplepulses and at pixel locations until the end of the frame period when aread of the charge amount occurs, and then the capacitor is reset. Thereare no read and reset operations during each small exposure that isturned on during a single pulse of the multiple pulses so that charge isaccumulated at individual pixel locations. Note that other methods maybe used to affect the same result of collecting charge only when thepulsed light is active and not when the pulsed light is inactive.

By one example form, the sensors may be complementarymetal-oxide-semiconductor-type image sensor (CMOS) infra-red (IR)detecting sensors optimized to have good response at 940 nm, Note thatthe methods herein may work with many image sensors capable ofdelivering acceptable performance when imaging at 940 nm. In the exampleherein, there are manufacturers that have developed a sensor with verygood quantum efficiency in the NIR range, and particularly at about 940nm detection with novel and proprietary techniques. Various techniquesmay also be used to improve the overall light collection efficiency.These techniques may include the use of stacked dies or layers, whereeach die or layer is optimized for the specific tasks. One die or layermay be optimized for optical conversion efficiency while another forelectronic control and readout out.

As to the projector, by one possible example, a VCSEL projector is usedand that emits light vertically or perpendicular to the face of thediode, and in turn the circuit board upon which it the diode sits,rather than to the side as in conventional lasers that require a prismto redirect the light in a direction perpendicular to the circuit board.The VCSEL projector provides a wide latitude for safety (where the powerused to attain the 940 nm light emission is well below the safety limitsas explained below). In one example, the pulse width may bepre-selected, and the frequency may be adjusted for a pre-selected inputpower or alternatively a pre-selected optical power. The actualfrequency needed will be determined depending on the componentvariations. The analog components for the laser system for the VCSEL maybe fail safe so that the laser cannot be operated in an unsafecondition. The VCSELs also perform more reliably than conventionallasers and LEDs. VCSEL projectors operating at 940 nm are available froma variety of companies including Princeton Optronics and Vixar.

It should be noted that other lasers and sensors could work. SiliconCMOS sensors, the most prevalent, typically do not have a sufficientresponse at desirable longer wavelengths at more than the 850 nm forexample, and conversely shorter wavelengths into the visible range wouldaffect the visible RGB sensor. Other types of sensors may be used forpattern light detection to obtain wavelengths other than 940 nm thathave a low sunlight energy. Also, other technologies exist such as LEDSand lasers that are not necessarily VCSEL technologies but that stillmay be used to provide the projector. These other types of lasers shouldhave features to ensure safe operation as required by FDA certification(e.g. the laser should be tolerant of single point failures), and soforth.

Once the light is detected, or more specifically a light detectioncharge is collected, the pattern sensors will convert this charge toelectrical signals and a depth calculation unit 420 may use this data.Then depth calculation unit 420 may perform tasks such as featurematching of points of two images from the two (or more) sensors, andthen depth calculations to obtain the (X, Y, Z) 3D space pixelcoordinates to form a 3D space. The depth calculation unit 420 then mayprovide a depth map for further processing.

Referring now to FIG. 5, an example process 500 for automatic exposurecontrol and pattern projection for cameras described herein is arrangedin accordance with at least some implementations of the presentdisclosure. In the illustrated implementation, process 500 may includeone or more operations, functions or actions as illustrated by one ormore of operations 502-516 numbered evenly. By way of non-limitingexample, process 500 may be described herein with reference to exampleimage processing system of FIGS. 1, 1A, 4, and 13 where relevant.

Continuing now with process 500, process 500 may include “initiate depthmeasurement” 502, which may be processes that are activated whenever animage is being captured by an imaging device, or may occur when a userhits a certain button or sets certain settings on the imaging device. Itwill be understood that the use of multiple exposures in a single frameperiod on a sensor may be used for other purposes other than depthmeasurement, and the operations described herein merely provides oneexample of its use.

Process 500 may include “obtain pulse parameters” 504. For setting thepulse parameters, the parameters are pulse amplitude, pulse width, andfrequency. These can be adjusted to obtain a desired overall averagepower level. For a VCSEL laser, this laser is most efficient when drivenbetween 6 amps and 20 amps. So for a start, there is a range of bestamplitude in which the laser should be operated. Once the amplitude isset, it is fixed, and may be fixed for the image capture device goingforward, but otherwise may be fixed for other periods (as long as thecamera is on, or per frame, for example).

Before continuing with process 500 to perform the light patterndetection, however, a few preliminary functions are, or were, performedduring manufacture of the image capture device for example, and asexplained herein to set the pulse parameters in the first place.Particularly, the projector was selected to emit a certain wavelength,and the pulse rate may have been pre-determined as well. The details forthis selection and setting are as follows.

As mentioned with the processes disclosed herein, one goal is tomaximize the exposure component of light of the projected pattern versusthe background light. In that sense, the processes herein attempt tomaximize the sensitivity to the pattern (such as pattern 200 of FIG. 2).This may be referred to as improving the Signal-to-Noise ratio (SNR orS/N ratio), where the projected pattern is the desired signal and thebackground light is the noise.

In more detail, sunlight rejection is a challenge for all active depthcameras because laser projection systems typically cast about 850 nmlight which has been found to be the best compromise between sensorresponse, laser efficiency, and eye safety on many conventional systems.This raises a large problem because the sun has a strong 850 nm contentand air is relatively transparent to 850 nm light (i.e., there is lessabsorption of the light in the atmosphere at 850 nm).

This cannot be solved simply by only decreasing the exposure time.Reducing the exposure time reduces the sensitivity to the projectedsignal. In other words, the sensitivity of the sensor (camera) is afixed quantity. A sensor with higher sensitivity would require a shorterexposure for a given intensity of light, versus one with a lowersensitivity. With too much exposure, the sensor is saturated and reachesa maximum value. With too little exposure, the sensor remains at aminimum value and does not sense the incoming light.

This also cannot be solved simply by increasing the projector power sothat the projected pattern cannot be washed out by sunlight because itraises both (1) thermal issues, and (2) eye safety limits. Regarding thethermal problems, simply stated, if one only increased the laser power,it is possible to overcome sunlight. But this would require anenormously powerful laser, which most likely will not be eye safe, andit would most likely dissipate a lot of heat (it may also require a verylarge battery for mobile applications).

The solution provided here improves the SNR for an image capture system,and that may use an assisted-3D stereoscopic system that uses anadaptive exposure modulated shutter CMOS sensor to improve itsrobustness to ambient illumination and specifically to improve itsoutdoor operation by at least 10 times.

Referring to FIG. 6, one way to increase the SNR is to select a lightwavelength that has less radiation (less energy) by sunlight so that itwill not be so easily washed out by the sunlight. A solar radiation atsea level chart 600 is provided to show the different energy (or solarirradiation) at different wave lengths. As indicated, 940 nm light has amuch lower (by 5×) radiation than at 850 nm which is one of wavelengthsused by conventional IR projectors. Specifically, the solar radiationchart shows that 940 nm wavelengths have sunlight (solar irradiation)reduced 5× (0.05 to 0.01 Hλ(Wm-2 Å-1)). In other words, sunlightincludes 5 to 10 times less radiation (or less energy or more atmosphereabsorption of the energy or sunlight) at 940 nm versus 850 nm. So bytuning the system to work at 940 nm, the system can improve the signalto noise ratio as mentioned herein. This suggests the use of a 940 nmlaser (or LED or VCSEL) and to have the sensor more sensitive to 940 nmlight (versus other wavelengths). It may also include adding appropriateoptical filters when needed in order to attain the 940 nm lightemission.

Another way to boost the SNR is to pulse the light pattern projection atsome duty cycle and frequency, and during a single frame period. A frameperiod is defined as the time required to complete a read operation ofthe sensor, or the time it takes to capture a single image, as mentionedabove. By pulsing the laser, the process or system maximizes theefficiency of the system especially when the laser is synchronized tothe exposure so that the laser is off while the exposure is off asexplained below. When the exposure time is reduced and the laser pulseenergy is increased, this improves the S/N ratio until some practicallimitation is reached based on factors such as the speed at which thesensor can gate the exposure time, speed at which the laser drivingcircuit can efficiently provide laser pulses, and eye safety concerns.

To state it another way, the method and system disclosed herein revolvesabout the ability for the laser driving circuit to efficiently providelaser pulses taking into account reasonable pulse power, input voltage,component sizes, and costs, as well as eye safety. By one form, acapacitor may be used to store energy used to pulse the laser. By havinga series of smaller pulses throughout the frame time instead of a singlepulse at the beginning of the frame time, the size of the capacitor isgreatly reduced. Not doing this results in an impractically (for amobile form factor) large capacitor. So the multiple pulses per frametime provides three advantages: (1) the pulse circuit uses a smallcapacitor, (2) the S/N ratio is improved by limiting the exposures tooccur during the pulse durations as described below, and (3) eye safetyis better because the energy in each pulse is much smaller (even if thepeak amplitude is higher) than a single pulse scenario so that thesystem complies with a single pulse MPE limit. (2) and (3) are discussedin detail below.

Using sufficiently short pulses at high intensity also improves theefficiency of a laser light source (e.g., overcomes threshold currentlosses). For “threshold current losses”, the laser does not produce any“laser light” until a threshold current is reached. The amount ofcurrent up to the threshold current is essentially wasted as heat. Anyadditional current is converted into light at some conversion efficiency(often stated watts/amp). So the light out is roughly:

Light Output=(Current Input−Threshold Current)*Conversion Efficiency

So it is desirable for the input current to be significantly higher thanthe threshold current to have a good overall power efficiency.

Next, the pulse width may be set. In the case of a particular driverimplementation, it has been found to be most efficient in producingpulses with pulse widths in the 250-800 ns range. The exact pulse may bedetermined with experimentation and varying the many parameters of thespecific components.

Also, the frequency may be set to achieve the desired power. In anycase, the system must attempt to ensure that the pulses (single or thetrain of pulses) meet the class 1 eye safety limit. Specifically, theeye safety threshold may be based on other factors in addition to thefrequency of the IR light projection pulses. The specification of theInternational Standard, IEC60825-1, Safety of Laser Products, edition1.2 (1993, 1997, 2001), a table of Acceptable Emission Limits (AEL) anda table of Maximum Permissible Exposure (MPE) (p. 52) may all be used,see sections 8 and 9 particularly.

The MPE looks at the peak energy in a single pulse and the integral(i.e., total combined) value of a train of pulses (or in other words, asrelated by the average power value over a time period for example). Inthe present case, by shifting to the train of pulses under theappropriate conditions, it is safer to provide an average energy over aframe period that has energy peaks that would not be acceptable for asingle pulse pattern.

To determine the safety threshold, first the applicable number ofpulses, the applicable duration of exposure, and a number of constantsis determined as described in the chart below and the IEC standardmentioned above, and these values are then plugged into an equation asthat shown below:

Exposure 1.8 × 5 × time t in s 10⁻⁵ 10⁻⁵ Wave- 10⁻¹³ 10⁻¹¹ 10⁻⁹ 10⁻⁷ toto 1 × 10⁻³ 10 10² 10³ 10⁴ length to to to to 5 × 1 × to to to to to λin nm 10⁻¹¹ 10⁻⁸ 10⁻⁷ 1.8 × 10⁻⁵ 10⁻⁵ 10⁻³ 10 10² 10³ 10⁴ 3 × 10⁴ 180 3× 10¹⁰ W · m⁻² 30 J · m⁻² to 302.5 C₂ J · m⁻² C₂ J · m⁻² 302.5 (t > T₁)to 315 (t ≦ T₁) 315 C₁ J · m⁻² 10⁴ J m⁻² 10 W m⁻² to 400 400 1.5 × 2.7 ×5 × 18 t^(0.75) C₆ J · m⁻² 400 to Retinal photochemical hazard to 700⁴10⁻⁴ C₆ 10⁴ t^(0.75) C₆ 10⁻³ C₆ 600 nm^(d) 100 C₃ 1 C₃ 1 C₃ J · m⁻² J ·m⁻² J · m⁻² J · m⁻² W · m⁻² W · m⁻² using using using γ_(p) = 11 mradγ_(p) = 1.1 t^(0.5) mrad γ_(p) = 110 mrad AND³ 400 to Retinal thermalhazard 700 nm^(d) a ≦ 1.5 mrad: 10 W · m⁻² a > 1.5 mrad; 18 C₆ T₂^(−0.25) W · m⁻² (t > T₂) (t ≦ T₂) 18 t^(0.75) C₆ J · m⁻² 700 1.5 × 2.7× 5 × 18 t^(0.75) a ≦ 1.5 mrad: 10 C₄ C₇ W · m⁻² to 1 050 10⁻⁴ C₄ C₆ 10⁴t^(0.75) C₄ C₆ 10⁻³ C₄ C₆ C₄ C₆ a > 1.5 mrad: 18 C₄ C₆ T₂ ^(−0.25) J ·m⁻² J · m⁻² J · m⁻² J · m⁻² W · m⁻² (t > T₂) 1 050 1.5 × 2.7 × 5 × 10⁻²C₆ C₇ J · m⁻² 90 t^(0.75) C₆ C₇ J · m⁻² (t ≦ T₂) to 1 400 10⁻³ C₆ C₇ 10⁵t^(0.75) C₆ C₇ 18 t^(0.75) C₄ C₆ C₇ J · m⁻² J · m⁻² J · m⁻² 1 400 10¹² W· m⁻² 10³ J · m⁻² 5 600 t^(0.25) J · m⁻² to 1 500The relevant equations from the above chart for NIR wave lengths 700 to1,050 are shown for the exposure time from 10 to (3×10⁴) columns.According to the specification, Class 1 is a Laser that is safe underreasonably foreseeable conditions of operation, including the use ofoptical instruments for intra beam viewing. In other words, if this typeof laser is pointed at a person, it will not injure the person. Athigher levels of power, the laser could be classified as Class 3x orabove (class 2 is reserved for visible wavelengths), which may behazardous. The class 1 lasers are used here. It should be noted that adifferent equation may be used to obtain the peak safety thresholdsdepending on the pulse width.

By one example form, a 200 kHz average power per frame period thresholdwas determined by using a RealSense™ R200 laser projector, but otherprojectors may have very different limit values. Specifically, and insimplified terms, it was determined that with laser modulation withpulse frequency <200 kHz, the eye safety threshold applies to the peak,not the average power over a frame period. However, when a laser ismodulated at a pulse frequency >200 khz (so that the pulses are closerto each other), the threshold peak power can be increased inversely andproportionally to the duty cycle (percentage of frame period with totalpulse duration) while maintaining required average power:

D=T/P

T=time the signal is active (on) and P is the total period of the signal(here frame period)

Power p increase=1/D=P/T

For example (just using random numbers for the example), if T/P=0.3 (thepulses are on about ⅓ of a frame period), then the peak power may beincreased by 3.33. Of course, there still may be a threshold for averagepower per frame period that must be complied with. Thus, shuttermodulation mitigates eye safety enabling significant increase in peakpower. This conclusion can be exemplified by the following examples inFIGS. 7-12.

Referring to FIG. 7, a power level graph 700 shows thermal limitationsfor a continuous wave operation. Thus, graph 700 shows that theprojector power must continuously remain below a continuous wave (CW)eye safety limit, which is below the sunlight power equivalent.

Referring to FIG. 8, a power level graph 800 shows eye safety limits fora single pulse operation. As shown, the pulse may undesirably equal orexceed the CW eye safety limit when the limit is based on peak projectorpower so that such a configuration would not pass eye safety approvaltests. It should be noted that the single pulse alternative may have theequivalent energy (same direct current (DC)) as a continuous laser.

Referring to FIG. 9, a power level graph 900 shows an average powerlimitation with a modulated operation of the projector power, andspecifically, in pulses grouped together in one part of the frame periodleaving a long non-pulsed section for the frame period for this example.As explained herein, this results in a significantly reduced averagepower over the frame period due to the 0 power duration that providesbalance against the higher power pulses. While not shown, it isunderstood that the actual average power is below the average eye safetylimit that is shown. As to the laser power levels used to obtain thedesired wavelengths, such as 940 nm, by one form, IEC60825 standarddiscloses that the power is the same for range from 700 nM to 1050 nMlight, and the same calculations are applied. Thus, for the differencein power to obtain 850 nm or 940 nm, there may not be an expecteddifference in power due to wavelength. It is the pulse configuration(and frequency) that will decide which power (peak or average or othervalue over a frame period) that is key to satisfying the eye safetystandards.

Some rough power values may be set similar to the following example. Ifit is assumed 0.150 W total output power is Class 1 safe, and a 50 KHzrepetition rate (frequency), each pulse would be 3E-6 W (e.g.0.150/50000). In terms of energy, if it is assumed each pulse is 1E-6 S,then the pulse energy is 3E-12 Joules (or 3 pico Joules). Again, thesevalues may be very different depending on the laser and other factorsmentioned herein. In one experiment, a request for power input (notoptical output) was found to be around 400 mW. For this, a prototype wasrunning around 25 Khz with a 375 nS pulse. This should provide a veryhigh safety margin.

Optionally, the desired power might be set lower than the eye safetylimit in order to limit the drain on the battery if it is a portabledevice, to limit the skin temperature of the product, or some otherfactor. The power might be lowered when the full range of the depthcamera is not needed, or when the camera is being used indoors to name afew examples.

Also as mentioned above, the pulse parameters, or pulse rate or pattern,per frame period may be pre-set and fixed on the devices (andparticularly at the one or more pulse generators), may be calculated atthe pulse generator, or may be received by a communication link fromanother component remote from the projector or remote from the devicethe projector is part of, or other options as may be contemplated, someof which are mentioned for system 400 above, and may include whether theprojector is the master or the camera/sensors is the master as alreadydiscussed above as well. In one form, adjustment of the pulse width andfrequency are performed digitally so that they are easy adjustments tomake, and once set, the amplitude may be fixed.

The process 500 then may include “set exposure time settings” 506, andthis may include “synchronize exposure by setting exposure start andfinish to correspond to start and finish of individual pulses” 508.Particularly, in order to increase the SNR the CMOS sensor could betriggered to expose a fraction of a frame time (or frame period), andthen burst the IR Projector at substantially the same time.

The reason to provide the synchronized exposure times is as follows.Whenever the sensor is in the mode to capture light (exposure is on),and the laser is on, then it is capturing sunlight and laser light. Whenthe laser is off, the sensor still may be capturing sunlight (unless ithas a physical shutter as well), but now only sunlight is being capturedwhich may add to the charge being collected by the sensor throughout theframe period. Countermeasures to this are referred to as sunlightrejection. So in order to reduce the capture of sunlight and improvesunlight rejection, it is best if there is no exposure when the laser isoff.

In the present system and methods, by altering the exposure mechanism ofa sensor system, it is possible to synchronize a series of shortduration light pulses with the exposure of the sensor throughout theframe time and to achieve the same result as a single, longer pulse atthe beginning of the frame. This significantly relaxes the driverequirements and eye safety concerns for the laser IR projector, such asa pulsed VCSEL, LED, or laser projector.

As the exposure is shortened, which can be a single exposure time perframe as a conventional global shutter camera, or multiple exposures perframe, as in this present process, while increasing the energy in theprojected image, the contribution to the background light from sunlightor other external lights is reduced while maintaining the contributionof the projected light.

For instance, referring to FIG. 10, a timeline graph 1000 illustratesthe conventional case using a pulsed light IR projector with a fullframe exposure. In this case, daylight (noise) is integrated over thefull frame time 1002. The graph 1000 shows frame, exposure gate, and IRlight pulse frequencies, and with light pulses 1004 emitted at a uniformfrequency along the entire frame period 1002. This results in anundesirable low SNR since a substantial amount of light contributes tothe collected charge between the projected light pulses.

Referring to FIG. 11, a timeline graph 1100 “enhanced S/N with adaptableexposure” provides a first attempted solution. By bunching up all thepulses at the beginning of the frame to form a continuous exposure as asingle large pulse 1104 in one part of a single frame period 1102, thesignal to noise ratio can be improved by a factor of A/B where A is theframe period and B is the total exposure duration performed during thelarge pulse 1102 and by the exposure gate on FIG. 11. The single pulseis formed by eliminating the off periods between light pulses to form acontinuous on period. However, this pulse pattern increases theperformance requirements of the laser and the laser drive circuit, andmay result in a pulse that does not meet the eye safety peak thresholdas mentioned above for chart 800 (FIG. 8).

Referring to FIG. 12, a timeline graph 1200 “enhanced S/N with adaptableexposure” illustrates the agile or adaptable multiple exposureconfiguration where the exposure times are synchronized to the lightpulses. Here, the same or similar signal-to-noise ratio is achievedwithout the need for a long pulse duration. In this alternative, auniform frequency of light pulses may be used along the entire frameperiod 1202 with the exposure gate turning the exposure on and off indurations 1206 that correspond to the pulses 1204.

The methods and system herein present at least two options. First, anenhanced S/N with adaptable exposure option (FIG. 12) provides a pulseconfiguration with a substantially uniform frequency for the pulsesthroughout the entire frame period. This option may enable achievementof the highest overall power while remaining eye safe, and it also maybe easier on a pulse circuit. Second, another pulse configuration (FIG.9) bunches the pulses in the beginning of the frame period, and with ahigher power, and omits the pulses in a large section of the frameperiod in order to maintain an overall average power level for the frameperiod. This second option may reduce motion blur and be better for anobject that is moving.

As to the duration of each exposure time, the exposure times may be setto start several ns before the start of the pulse, and end several nsafter the end of the pulse to better ensure all of the projected lightis efficiently detected. By other alternatives, the exposure time may beset to start and end at the same ns setting as the light pulse tomaximize sunlight rejection. By one alternative, the exposures may beset around 375 ns.

The system also may provide tables to select frequencies that correspondto low, medium, or high power, as an example, depending upon the needsof the systems. For example, tables may have different settings forindoor close range, indoor long range, or outdoor maximum performancefor example. Application software or middleware might select which powerlevel, or pulse pattern best suits the camera's needs.

While process 500 shows the pulse parameter determination before theexposure time setting as in a system where the light source is themaster and the camera is the slave, these operations could be in reverseorder where the camera is the master with exposure settings that arethen used to determine pulse frequencies to match exposure times fromthe settings. It will be understood of course that the camera or sensorsstill may be considered the master with the task of setting the pulsesand thereafter setting the exposure times based on the light pulses andthen transmitting the pulse parameters to the slave projector.

Completing process 500, process 500 may include “project patternaccording to pulse configuration” 510, and as already described above.The projection may provide a rectangle, or other shaped pattern, as inlight pattern 200 (FIG. 2) to provide artificial features to objects ina scene being captured and that has little or no features such as ablank white wall to name an example. Note that the projection may beused on scenes that do have features, or may be set to project no matterthe contents of a scene.

Process 500 may include “operate shutter according to synchronizedexposure time settings” 512, and as already explained with system 400for example. Thus, the shutter is operated to perform exposure timedurations that correspond to the light pulse durations as explainedherein.

Then, process 500 may include “detect pattern” 514, and by the sensorsas already explained, and then may include “perform depth calculations”516, and when the process is to perform 3D locating of objects in ascene being viewed or photographed by a camera (or device with one ormore cameras) triangulation computations may be performed to determinethe distance from the sensor to the projected pattern an object.

While many of the implementation and applications herein is for astereoscopic 3D camera, the features described herein are notnecessarily limited to stereoscopic or even to the 3D camera. Besidesstereoscopic, the technique may be applied to structured light, codedlight, or any other type of depth camera dependent on detecting aprojected light pattern. Besides depth cameras, this method may beuseful in any situation where one needs the auxiliary light source todominate over the ambient light to detect the auxiliary light, such asface and iris detection systems. Thus, any of these or similar systemsthat project light and have an exposure control may use the benefit ofsynchronizing the exposure times to the pulses so that a single pulseexposure time corresponds to a single pulse duration for multipleindividual pulses during a single frame period and as described indetail herein.

Thus, it also will be appreciated that the method and system ofsynchronized light pulses and exposure times during a single frameperiod may be applicable to other sensors and projectors with lightother than in the NIR range.

In addition, any one or more of the operations of FIGS. 2-4 and 6-7 maybe undertaken in response to instructions provided by one or morecomputer program products. Such program products may include signalbearing media providing instructions that, when executed by, forexample, a processor, may provide the functionality described herein.The computer program products may be provided in any form of one or moremachine-readable media. Thus, for example, a processor including one ormore processor core(s) may undertake one or more of the operations ofthe example processes herein in response to program code and/orinstructions or instruction sets conveyed to the processor by one ormore computer or machine-readable media. In general, a machine-readablemedium may convey software in the form of program code and/orinstructions or instruction sets that may cause any of the devicesand/or systems to perform as described herein. The machine or computerreadable media may be a non-transitory article or medium, such as anon-transitory computer readable medium, and may be used with any of theexamples mentioned above or other examples except that it does notinclude a transitory signal per se. It does include those elements otherthan a signal per se that may hold data temporarily in a “transitory”fashion such as RAM and so forth.

As used in any implementation described herein, the term “module” refersto any combination of software logic and/or firmware logic configured toprovide the functionality described herein. The software may be embodiedas a software package, code and/or instruction set, and/or firmware thatstores instructions executed by programmable circuitry. The modules may,collectively or individually, be embodied for implementation as part ofa larger system, for example, an integrated circuit (IC), system on-chip(SoC), and so forth.

As used in any implementation described herein, the term “logic unit”refers to any combination of firmware logic and/or hardware logicconfigured to provide the functionality described herein. The“hardware”, as used in any implementation described herein, may include,for example, singly or in any combination, hardwired circuitry,programmable circuitry, state machine circuitry, and/or firmware thatstores instructions executed by programmable circuitry. The logic unitsmay, collectively or individually, be embodied as circuitry that formspart of a larger system, for example, an integrated circuit (IC), systemon-chip (SoC), and so forth. For example, a logic unit may be embodiedin logic circuitry for the implementation firmware or hardware of thesystems discussed herein. Further, one of ordinary skill in the art willappreciate that operations performed by hardware and/or firmware mayalso utilize a portion of software to implement the functionality of thelogic unit.

As used in any implementation described herein, the term “engine” and/or“component” may refer to a module or to a logic unit, as these terms aredescribed above. Accordingly, the term “engine” and/or “component” mayrefer to any combination of software logic, firmware logic, and/orhardware logic configured to provide the functionality described herein.For example, one of ordinary skill in the art will appreciate thatoperations performed by hardware and/or firmware may alternatively beimplemented via a software module, which may be embodied as a softwarepackage, code and/or instruction set, and also appreciate that a logicunit may also utilize a portion of software to implement itsfunctionality.

Referring to FIG. 13, an example image processing system 1300 isarranged in accordance with at least some implementations of the presentdisclosure. In various implementations, the example image processingsystem 1300 may have an imaging device 1302 to form or receive capturedimage data as well as to detect light patterns used to providedetectable points to featureless objects. This can be implemented invarious ways. Thus, in one form, the image processing system 1300 may bea digital camera or other image capture device, and imaging device 1302,in this case, may be the camera hardware and camera sensor software,module, or component 1312. In other examples, imaging processing system1300 may have an imaging device 1302 that includes or may be one or morecameras with one or more pattern sensors, and logic modules 1304 maycommunicate remotely with, or otherwise may be communicatively coupledto, the imaging device 1302 for further processing of the image data.

A light source 1311, such as a pattern projector as described herein,may be in the form of a laser, LED, VCSEL, or other such projector thatmay project a pattern detectable by the sensors of the imaging devices,and which may project in non-visible light wavelengths such IR or NIR.The projector may be considered part of an imaging device or devices, ormay be separate from the devices. Thus, the system 1300 may be a smartphone with two imaging devices (two cameras) and a separate light source1311 all mounted on the same system (smart phone, tablet, digitalcamera, or other such device).

By more examples, such technology may include a camera such as a digitalcamera system, a dedicated camera device, or an imaging phone, whether astill picture or video camera or some combination of both. Thus, in oneform, imaging device 1302 may include camera hardware and opticsincluding one or more sensors 1306 as well as auto-focus, zoom,aperture, ND-filter, auto-exposure, flash, and actuator controls. Thesecontrols may be part of a sensor module or component 1306 for includingand operating sensors, including one or more IR (or other light) orpattern detecting sensor, which may be a CMOS sensor. The sensorcomponent 1306 may be part of the imaging device 1302, or may be part ofthe logical modules 1304 or both. Such sensor component can be used tocontrol the pattern sensors to detect a projected pattern and providedata indicating the position of the pattern relative to the sensor aswell as to generate images for a viewfinder and/or record still picturesor video. The imaging device 1302 also may have a lens, an image sensorwith a RGB Bayer color filter, an analog amplifier, an A/D converter,other components to convert incident light (whether visible light ornot) into a digital signal, the like, and/or combinations thereof. Thedigital signal also may be referred to as the raw image data herein.

Other forms include a camera sensor-type imaging device or the like (forexample, a webcam or webcam sensor or other complementarymetal-oxide-semiconductor-type image sensor (CMOS)), with or without theuse of a red-green-blue (RGB) depth camera, infra-red (IR) sensor asdescribed herein, and/or microphone-array to locate who is speaking. TheIR camera sensor is a global shutter, while other camera sensor(s) maysupport other types of electronic shutters, such as global shutter orrolling shutter or both, and many other shutter types. In otherexamples, an RGB-Depth camera may have an IR camera sensor. In someexamples, imaging device 1302 may be provided with an eye trackingcamera as well.

In the illustrated example, the logic modules 1304 may include theautomatic white balancing control 1314, automatic focus (AF) module1316, and exposure control component 1316 to control sensor exposure. Apattern control unit 1314 may be provided to control the light source1311. A pattern synchronization unit 1310 or 1332 may communicate withor may include a pulse generator 1336 that may or may not be on-board anISP 1322, a microcontroller, or other processor 1320. The pulsegenerator 1336 may be considered part of the pattern control unit, theexposure control component, any combination of these or none of thesesuch that the pulse generator 1336 may be considered to be separate fromthe camera and light source. The pulse generator 1336 may be connectedto a communication link for receiving pulse configuration data, and maybe in communication with an interface for such purpose. The logicmodules may be communicatively coupled to the imaging device 1302 inorder to receive the pattern detection data as well as other raw imagedata.

The image processing system 1300 may have one or more processors 1320which may include the dedicated image signal processor (ISP) 1322 suchas the Intel Atom, memory stores 1324, one or more displays 1326,encoder 1328, and antenna 1330. In one example implementation, the imageprocessing system 1300 may have the display 1326, at least one processor1320 communicatively coupled to the display, at least one memory 1324communicatively coupled to the processor and having a frame buffer 1334.The buffer as well as memory store(s) 1324 may store the frame data fromthe IR sensor(s) 1306 for use in depth calculations performed by thedepth calc. unit 1340 and the processor(s) 1320 or other uses.

The encoder 1328 and antenna 1330 may be provided to compress themodified image date for transmission to other devices that may displayor store the image. It will be understood that the image processingsystem 1300 may also include a decoder (or encoder 1328 may include adecoder) to receive and decode image data for processing by the system1300. Otherwise, the processed image 1332 may be displayed on display1326 or stored in memory 1324. As illustrated, any of these componentsmay be capable of communication with one another and/or communicationwith portions of logic modules 1304 and/or imaging device 1302. Thus,processors 1320 may be communicatively coupled to both the image device1302 and the logic modules 1304 for operating those components. By oneapproach, although image processing system 1300, as shown in FIG. 13,may include one particular set of blocks or actions associated withparticular components or modules, these blocks or actions may beassociated with different components or modules than the particularcomponent or module illustrated here.

Referring to FIG. 14, an example system 1400 in accordance with thepresent disclosure operates one or more aspects of the image processingsystem described herein. It will be understood from the nature of thesystem components described below that such components may be associatedwith, or used to operate, certain part or parts of the image processingsystem described above. In various implementations, system 1400 may be amedia system although system 1400 is not limited to this context. Forexample, system 1400 may be incorporated into a digital still camera,digital video camera, mobile device with camera or video functions suchas an imaging phone, webcam, personal computer (PC), laptop computer,ultra-laptop computer, tablet, touch pad, portable computer, handheldcomputer, palmtop computer, personal digital assistant (PDA), cellulartelephone, combination cellular telephone/PDA, television, smart device(e.g., smart phone, smart tablet or smart television), mobile internetdevice (MID), messaging device, data communication device, and so forth.

In various implementations, system 1400 includes a platform 1402 coupledto a display 1420. Platform 1402 may receive content from a contentdevice such as content services device(s) 1430 or content deliverydevice(s) 1440 or other similar content sources. A navigation controller1450 including one or more navigation features may be used to interactwith, for example, platform 1402 and/or display 1420. Each of thesecomponents is described in greater detail below.

In various implementations, platform 1402 may include any combination ofa chipset 1405, processor 1410, memory 1412, storage 1414, graphicssubsystem 1415, applications 1416 and/or radio 1418. Chipset 1405 mayprovide intercommunication among processor 1410, memory 1412, storage1414, graphics subsystem 1415, applications 1416 and/or radio 1418. Forexample, chipset 1405 may include a storage adapter (not depicted)capable of providing intercommunication with storage 1414.

Processor 1410 may be implemented as a Complex Instruction Set Computer(CISC) or Reduced Instruction Set Computer (RISC) processors; x86instruction set compatible processors, multi-core, or any othermicroprocessor or central processing unit (CPU). In variousimplementations, processor 1410 may be dual-core processor(s), dual-coremobile processor(s), and so forth.

Memory 1412 may be implemented as a volatile memory device such as, butnot limited to, a Random Access Memory (RAM), Dynamic Random AccessMemory (DRAM), or Static RAM (SRAM).

Storage 1414 may be implemented as a non-volatile storage device suchas, but not limited to, a magnetic disk drive, optical disk drive, tapedrive, an internal storage device, an attached storage device, flashmemory, battery backed-up SDRAM (synchronous DRAM), and/or a networkaccessible storage device. In various implementations, storage 1414 mayinclude technology to increase the storage performance enhancedprotection for valuable digital media when multiple hard drives areincluded, for example.

Graphics subsystem 1415 may perform processing of images such as stillor video for display. Graphics subsystem 1415 may be a graphicsprocessing unit (GPU) or a visual processing unit (VPU), for example. Ananalog or digital interface may be used to communicatively couplegraphics subsystem 1415 and display 1420. For example, the interface maybe any of a High-Definition Multimedia Interface, Display Port, wirelessHDMI, and/or wireless HD compliant techniques. Graphics subsystem 1415may be integrated into processor 1410 or chipset 1405. In someimplementations, graphics subsystem 1415 may be a stand-alone cardcommunicatively coupled to chipset 1405.

The graphics and/or video processing techniques described herein may beimplemented in various hardware architectures. For example, graphicsand/or video functionality may be integrated within a chipset.Alternatively, a discrete graphics and/or video processor may be used.As still another implementation, the graphics and/or video functions maybe provided by a general purpose processor, including a multi-coreprocessor. In further embodiments, the functions may be implemented in aconsumer electronics device.

Radio 1418 may include one or more radios capable of transmitting andreceiving signals using various suitable wireless communicationstechniques. Such techniques may involve communications across one ormore wireless networks. Example wireless networks include (but are notlimited to) wireless local area networks (WLANs), wireless personal areanetworks (WPANs), wireless metropolitan area network (WMANs), cellularnetworks, and satellite networks. In communicating across such networks,radio 818 may operate in accordance with one or more applicablestandards in any version.

In various implementations, display 1420 may include any television typemonitor or display. Display 1420 may include, for example, a computerdisplay screen, touch screen display, video monitor, television-likedevice, and/or a television. Display 1420 may be digital and/or analog.In various implementations, display 1420 may be a holographic display.Also, display 1420 may be a transparent surface that may receive avisual projection. Such projections may convey various forms ofinformation, images, and/or objects. For example, such projections maybe a visual overlay for a mobile augmented reality (MAR) application.Under the control of one or more software applications 1416, platform1402 may display user interface 1422 on display 1420.

In various implementations, content services device(s) 1430 may behosted by any national, international and/or independent service andthus accessible to platform 1402 via the Internet, for example. Contentservices device(s) 1430 may be coupled to platform 1402 and/or todisplay 1420. Platform 1402 and/or content services device(s) 1430 maybe coupled to a network 1460 to communicate (e.g., send and/or receive)media information to and from network 1460. Content delivery device(s)1440 also may be coupled to platform 1402 and/or to display 1420.

In various implementations, content services device(s) 1430 may includea cable television box, personal computer, network, telephone, Internetenabled devices or appliance capable of delivering digital informationand/or content, and any other similar device capable of unidirectionallyor bidirectionally communicating content between content providers andplatform 1402 and/display 1420, via network 1460 or directly. It will beappreciated that the content may be communicated unidirectionally and/orbidirectionally to and from any one of the components in system 1400 anda content provider via network 1460. Examples of content may include anymedia information including, for example, video, music, medical andgaming information, and so forth.

Content services device(s) 1430 may receive content such as cabletelevision programming including media information, digital information,and/or other content. Examples of content providers may include anycable or satellite television or radio or Internet content providers.The provided examples are not meant to limit implementations inaccordance with the present disclosure in any way.

In various implementations, platform 1402 may receive control signalsfrom navigation controller 1450 having one or more navigation features.The navigation features of controller 1450 may be used to interact withuser interface 1422, for example. In embodiments, navigation controller1450 may be a pointing device that may be a computer hardware component(specifically, a human interface device) that allows a user to inputspatial (e.g., continuous and multi-dimensional) data into a computer.Many systems such as graphical user interfaces (GUI), and televisionsand monitors allow the user to control and provide data to the computeror television using physical gestures.

Movements of the navigation features of controller 1450 may bereplicated on a display (e.g., display 1420) by movements of a pointer,cursor, focus ring, or other visual indicators displayed on the display.For example, under the control of software applications 1416, thenavigation features located on navigation controller 1450 may be mappedto virtual navigation features displayed on user interface 1422, forexample. In implementations, controller 1450 may not be a separatecomponent but may be integrated into platform 1402 and/or display 1420.The present disclosure, however, is not limited to the elements or inthe context shown or described herein.

In various implementations, drivers (not shown) may include technologyto enable users to instantly turn on and off platform 1402 like atelevision with the touch of a button after initial boot-up, whenenabled, for example. Program logic may allow platform 1402 to streamcontent to media adaptors or other content services device(s) 1430 orcontent delivery device(s) 1440 even when the platform is turned “off.”In addition, chipset 1405 may include hardware and/or software supportfor 8.1 surround sound audio and/or high definition (7.1) surround soundaudio, for example. Drivers may include a graphics driver for integratedgraphics platforms. In embodiments, the graphics driver may comprise aperipheral component interconnect (PCI) Express graphics card.

In various implementations, any one or more of the components shown insystem 1400 may be integrated. For example, platform 1402 and contentservices device(s) 1430 may be integrated, or platform 1402 and contentdelivery device(s) 1440 may be integrated, or platform 1402, contentservices device(s) 1430, and content delivery device(s) 1440 may beintegrated, for example. In various embodiments, platform 1402 anddisplay 1420 may be an integrated unit. Display 1420 and content servicedevice(s) 1430 may be integrated, or display 1420 and content deliverydevice(s) 1440 may be integrated, for example. These examples are notmeant to limit the present disclosure.

In various embodiments, system 1400 may be implemented as a wirelesssystem, a wired system, or a combination of both. When implemented as awireless system, system 1400 may include components and interfacessuitable for communicating over a wireless shared media, such as one ormore antennas, transmitters, receivers, transceivers, amplifiers,filters, control logic, and so forth. An example of wireless sharedmedia may include portions of a wireless spectrum, such as the RFspectrum and so forth. When implemented as a wired system, system 1400may include components and interfaces suitable for communicating overwired communications media, such as input/output (I/O) adapters,physical connectors to connect the I/O adapter with a correspondingwired communications medium, a network interface card (NIC), disccontroller, video controller, audio controller, and the like. Examplesof wired communications media may include a wire, cable, metal leads,printed circuit board (PCB), backplane, switch fabric, semiconductormaterial, twisted-pair wire, co-axial cable, fiber optics, and so forth.

Platform 1402 may establish one or more logical or physical channels tocommunicate information. The information may include media informationand control information. Media information may refer to any datarepresenting content meant for a user. Examples of content may include,for example, data from a voice conversation, videoconference, streamingvideo, electronic mail (“email”) message, voice mail message,alphanumeric symbols, graphics, image, video, text and so forth. Datafrom a voice conversation may be, for example, speech information,silence periods, background noise, comfort noise, tones and so forth.Control information may refer to any data representing commands,instructions or control words meant for an automated system. Forexample, control information may be used to route media informationthrough a system, or instruct a node to process the media information ina predetermined manner. The implementations, however, are not limited tothe elements or in the context shown or described in FIG. 14.

Referring to FIG. 14, a small form factor device 1400 is one example ofthe varying physical styles or form factors in which system 1400 may beembodied. By this approach, device 1400 may be implemented as a mobilecomputing device having wireless capabilities. A mobile computing devicemay refer to any device having a processing system and a mobile powersource or supply, such as one or more batteries, for example.

As described above, examples of a mobile computing device may include adigital still camera, digital video camera, mobile devices with cameraor video functions such as imaging phones, webcam, personal computer(PC), laptop computer, ultra-laptop computer, tablet, touch pad,portable computer, handheld computer, palmtop computer, personal digitalassistant (PDA), cellular telephone, combination cellular telephone/PDA,television, smart device (e.g., smart phone, smart tablet or smarttelevision), mobile internet device (MID), messaging device, datacommunication device, and so forth.

Examples of a mobile computing device also may include computers thatare arranged to be worn by a person, such as a wrist computer, fingercomputer, ring computer, eyeglass computer, belt-clip computer, arm-bandcomputer, shoe computers, clothing computers, and other wearablecomputers. In various embodiments, for example, a mobile computingdevice may be implemented as a smart phone capable of executing computerapplications, as well as voice communications and/or datacommunications. Although some embodiments may be described with a mobilecomputing device implemented as a smart phone by way of example, it maybe appreciated that other embodiments may be implemented using otherwireless mobile computing devices as well. The embodiments are notlimited in this context.

As shown in FIG. 15, device 1500 may include a housing 1502, a display1504 including a screen 1510, an input/output (I/O) device 1506, and anantenna 1508. Device 1500 also may include navigation features 1512.Display 1504 may include any suitable display unit for displayinginformation appropriate for a mobile computing device. I/O device 1506may include any suitable I/O device for entering information into amobile computing device. Examples for I/O device 1506 may include analphanumeric keyboard, a numeric keypad, a touch pad, input keys,buttons, switches, rocker switches, microphones, speakers, voicerecognition device and software, and so forth. Information also may beentered into device 1500 by way of microphone (not shown). Suchinformation may be digitized by a voice recognition device (not shown).The embodiments are not limited in this context.

Various forms of the devices and processes described herein may beimplemented using hardware elements, software elements, or a combinationof both. Examples of hardware elements may include processors,microprocessors, circuits, circuit elements (e.g., transistors,resistors, capacitors, inductors, and so forth), integrated circuits,application specific integrated circuits (ASIC), programmable logicdevices (PLD), digital signal processors (DSP), field programmable gatearray (FPGA), logic gates, registers, semiconductor device, chips,microchips, chip sets, and so forth. Examples of software may includesoftware components, programs, applications, computer programs,application programs, system programs, machine programs, operatingsystem software, middleware, firmware, software modules, routines,subroutines, functions, methods, procedures, software interfaces,application program interfaces (API), instruction sets, computing code,computer code, code segments, computer code segments, words, values,symbols, or any combination thereof. Determining whether an embodimentis implemented using hardware elements and/or software elements may varyin accordance with any number of factors, such as desired computationalrate, power levels, heat tolerances, processing cycle budget, input datarates, output data rates, memory resources, data bus speeds and otherdesign or performance constraints.

One or more aspects of at least one embodiment may be implemented byrepresentative instructions stored on a machine-readable medium whichrepresents various logic within the processor, which when read by amachine causes the machine to fabricate logic to perform the techniquesdescribed herein. Such representations, known as “IP cores” may bestored on a tangible, machine readable medium and supplied to variouscustomers or manufacturing facilities to load into the fabricationmachines that actually make the logic or processor.

While certain features set forth herein have been described withreference to various implementations, this description is not intendedto be construed in a limiting sense. Hence, various modifications of theimplementations described herein, as well as other implementations,which are apparent to persons skilled in the art to which the presentdisclosure pertains are deemed to lie within the spirit and scope of thepresent disclosure.

The following examples pertain to further implementations.

By one implementation, a method of adaptable exposure control and lightprojection for cameras comprises projecting light in multiple pulses ina single frame period, wherein the frame period is used to capture asingle image of a scene; performing multiple exposure times during thesingle frame period and at at least one sensor used to detect the scene;and synchronizing the exposure times to the pulses so that a singleexposure time corresponds to a single pulse duration for multipleindividual pulses during the single frame period.

By a further implementation, the method also may comprise projecting aninfra-red light; and emitting and sensing light at about 940 nm or about939 nm to about 941 nm; wherein at least one of: (1) the pulses areprovided at a substantially uniform frequency throughout the frameperiod, and (2) the pulses are concentrated so that at least onenon-pulsed duration exists during the frame period and that has aduration longer than a duration between adjacent pulses used multipletimes during the frame period, wherein the non-pulsed duration in aframe period is longer than the total duration of the pulses in theframe period, wherein the pulses are bunched together in a single areaof the frame period. Also, the method may include wherein the pulses areprovided so that the pulses of a frame period has an average power ofthe frame period that is below a threshold average power, but a peakpower of the pulses is above the threshold average power, wherein thethreshold is associated with a laser eye safety threshold; wherein allexposure times in a frame period are synchronized to a pulse during theframe period; and the method comprising receiving a signal from thesensors associated with the detection of the projected light, and usingthe signal to perform depth measurement calculations, wherein the lightis projected as a geometrical pattern.

By yet a further implementation, a system may comprise comprising: atleast one projector to project a light pattern to provide features for ascene; at least one sensor to detect the pattern; at least one memory;at least one processor communicatively and operatively coupled to theprojector, sensor, and memory; a light unit operated by the at least oneprocessor and arranged to control the projector to project light inmultiple pulses in a single frame period, wherein the frame period isused to capture a single image of a scene; an exposure unit operated bythe at least one processor and arranged to control the at least onesensor to perform multiple exposure times during the single frame periodand at at least one sensor used to detect the scene; and asynchronization unit operated by the at least one processor and arrangedto synchronize the exposure times to the pulses so that a singleexposure time corresponds to a single pulse duration for multipleindividual pulses during the single frame period.

By another implementation, the system may comprise wherein the lightunit is to project an infra-red light; and emit and sense light at about940 nm or about 939 nm to about 941 nm; wherein at least one of: (1) thepulses are provided at a substantially uniform frequency throughout theframe period, and (2) the pulses are concentrated so that at least onenon-pulsed duration exists during the frame period and that has aduration longer than a duration between adjacent pulses used multipletimes during the frame period, wherein the non-pulsed duration in aframe period is longer than the total duration of the pulses in theframe period, wherein the pulses are bunched together in a single areaof the frame period. Also, the system includes wherein the pulses areprovided so that the pulses of a frame period has an average power ofthe frame period that is below a threshold average power, but a peakpower of the pulses is above the threshold average power, wherein thethreshold is associated with a laser eye safety threshold; wherein allexposure times in a frame period are synchronized to a pulse during theframe period; and the system being arranged to receive a signal from thesensors associated with the detection of the projected light, and usethe signal to perform depth measurement calculations, wherein the lightis projected as a geometrical pattern.

By yet another implementation, a computer readable medium having storedthereon instructions that when executed cause a computing device to:project light in multiple pulses in a single frame period, wherein theframe period is used to capture a single image of a scene; performmultiple exposure times during the single frame period and at at leastone sensor used to detect the scene; and synchronize the exposure timesto the pulses so that a single exposure time corresponds to a singlepulse duration for multiple individual pulses during the single frameperiod.

By an additional implementation, the computer readable medium whereinthe instructions cause the computing device to: project an infra-redlight; and emit and sense light at about 940 nm or about 939 nm to about941 nm; wherein at least one of: (1) the pulses are provided at asubstantially uniform frequency throughout the frame period, and (2) thepulses are concentrated so that at least one non-pulsed duration existsduring the frame period and that has a duration longer than a durationbetween adjacent pulses used multiple times during the frame period,wherein the non-pulsed duration in a frame period is longer than thetotal duration of the pulses in the frame period, wherein the pulses arebunched together in a single area of the frame period. The instructionscause the computing device to include wherein the pulses are provided sothat the pulses of a frame period has an average power of the frameperiod that is below a threshold average power, but a peak power of thepulses is above the threshold average power, wherein the threshold isassociated with a laser eye safety threshold; wherein all exposure timesin a frame period are synchronized to a pulse during the frame period;and wherein the instructions cause the computing device to receive asignal from the sensors associated with the detection of the projectedlight, and use the signal to perform depth measurement calculations,wherein the light is projected as a geometrical pattern.

In a further example, at least one machine readable medium may include aplurality of instructions that in response to being executed on acomputing device, causes the computing device to perform the methodaccording to any one of the above examples.

In a still further example, an apparatus may include means forperforming the methods according to any one of the above examples.

The above examples may include specific combination of features.However, the above examples are not limited in this regard and, invarious implementations, the above examples may include undertaking onlya subset of such features, undertaking a different order of suchfeatures, undertaking a different combination of such features, and/orundertaking additional features than those features explicitly listed.For example, all features described with respect to any example methodsherein may be implemented with respect to any example apparatus, examplesystems, and/or example articles, and vice versa.

What is claimed is:
 1. A method of adaptable exposure control and lightprojection for cameras, comprising: projecting light in multiple pulsesin a single frame period, wherein the frame period is used to capture asingle image of a scene; performing multiple exposure times during thesingle frame period and at at least one sensor used to detect the scene;and synchronizing the exposure times to the pulses so that a singleexposure time corresponds to a single pulse duration for multipleindividual pulses during the single frame period.
 2. The method of claim1 comprising projecting an infra-red light.
 3. The method of claim 1comprising emitting and sensing light at about 940 nm or about 939 nm toabout 941 nm.
 4. The method of claim 1 wherein the pulses are providedat a substantially uniform frequency throughout the frame period.
 5. Themethod of claim 1 wherein the pulses are concentrated so that at leastone non-pulsed duration exists during the frame period and that has aduration longer than a duration between adjacent pulses used multipletimes during the frame period.
 6. The method of claim 5 wherein thenon-pulsed duration in a frame period is longer than the total durationof the pulses in the frame period.
 7. The method of claim 5 wherein thepulses are bunched together in a single area of the frame period.
 8. Themethod of claim 1 wherein the pulses are provided so that the pulses ofa frame period has an average power of the frame period that is below athreshold average power, but a peak power of the pulses is above thethreshold average power.
 9. The method of claim 8 wherein the thresholdis associated with a laser eye safety threshold.
 10. The method of claim1 wherein all exposure times in a frame period are synchronized to apulse during the frame period.
 11. The method of claim 1 comprisingreceiving a signal from the sensors associated with the detection of theprojected light, and using the signal to perform depth measurementcalculations.
 12. The method of claim 1 wherein the light is projectedas a geometrical pattern.
 13. The method of claim 1 comprisingprojecting an infra-red light; and emitting and sensing light at about940 nm or about 939 nm to about 941 nm; wherein at least one of: thepulses are provided at a substantially uniform frequency throughout theframe period, and the pulses are concentrated so that at least onenon-pulsed duration exists during the frame period and that has aduration longer than a duration between adjacent pulses used multipletimes during the frame period, wherein the non-pulsed duration in aframe period is longer than the total duration of the pulses in theframe period, wherein the pulses are bunched together in a single areaof the frame period; wherein the pulses are provided so that the pulsesof a frame period has an average power of the frame period that is belowa threshold average power, but a peak power of the pulses is above thethreshold average power, wherein the threshold is associated with alaser eye safety threshold; wherein all exposure times in a frame periodare synchronized to a pulse during the frame period; and the methodcomprising receiving a signal from the sensors associated with thedetection of the projected light, and using the signal to perform depthmeasurement calculations, wherein the light is projected as ageometrical pattern.
 14. A system comprising: at least one projector toproject a light pattern to provide features for a scene; at least onesensor to detect the pattern; at least one memory; at least oneprocessor communicatively and operatively coupled to the projector,sensor, and memory; a light unit operated by the at least one processorand arranged to control the projector to project light in multiplepulses in a single frame period, wherein the frame period is used tocapture a single image of a scene; an exposure unit operated by the atleast one processor and arranged to control the at least one sensor toperform multiple exposure times during the single frame period and at atleast one sensor used to detect the scene; and a synchronization unitoperated by the at least one processor and arranged to synchronize theexposure times to the pulses so that a single exposure time correspondsto a single pulse duration for multiple individual pulses during thesingle frame period.
 15. The system of claim 14 wherein the light is ofa projection of infra-red light.
 16. The system of claim 14 wherein thelight is emitted at about 940 nm or about 939 nm to about 941 nm. 17.The system of claim 14 wherein the pulses are provided at asubstantially uniform frequency throughout the frame period.
 18. Thesystem of claim 14 wherein the pulses are concentrated so that at leastone non-pulsed duration exists during the frame period and that has aduration longer than a duration between adjacent pulses used multipletimes during the frame period.
 19. The system of claim 18 wherein thenon-pulsed duration in a frame period is longer than the total durationof the pulses in the frame period.
 20. The system of claim 18 whereinthe pulses are bunched together in a single area of the frame period.21. The system of claim 14 wherein the pulses are provided so that thepulses of a frame period has an average power of the frame period thatis below a threshold average power, but a peak power of the pulses isabove the threshold average power.
 22. The system of claim 21 whereinthe threshold is associated with a laser eye safety threshold.
 23. Thesystem of claim 14 wherein the light unit is to project an infra-redlight; and emit and sense light at about 940 nm or about 939 nm to about941 nm; wherein at least one of: the pulses are provided at asubstantially uniform frequency throughout the frame period, and thepulses are concentrated so that at least one non-pulsed duration existsduring the frame period and that has a duration longer than a durationbetween adjacent pulses used multiple times during the frame period,wherein the non-pulsed duration in a frame period is longer than thetotal duration of the pulses in the frame period, wherein the pulses arebunched together in a single area of the frame period; wherein thepulses are provided so that the pulses of a frame period has an averagepower of the frame period that is below a threshold average power, but apeak power of the pulses is above the threshold average power, whereinthe threshold is associated with a laser eye safety threshold; whereinall exposure times in a frame period are synchronized to a pulse duringthe frame period; and the system being arranged to receive a signal fromthe sensors associated with the detection of the projected light, anduse the signal to perform depth measurement calculations, wherein thelight is projected as a geometrical pattern.
 24. A computer-readablemedium having stored thereon instructions that when executed cause acomputing device to: project light in multiple pulses in a single frameperiod, wherein the frame period is used to capture a single image of ascene; perform multiple exposure times during the single frame periodand at at least one sensor used to detect the scene; and synchronize theexposure times to the pulses so that a single exposure time correspondsto a single pulse duration for multiple individual pulses during thesingle frame period.
 25. The computer-readable medium of claim 24wherein the instructions cause the computing device to: project aninfra-red light; and emit and sense light at about 940 nm or about 939nm to about 941 nm; wherein at least one of: the pulses are provided ata substantially uniform frequency throughout the frame period, and thepulses are concentrated so that at least one non-pulsed duration existsduring the frame period and that has a duration longer than a durationbetween adjacent pulses used multiple times during the frame period,wherein the non-pulsed duration in a frame period is longer than thetotal duration of the pulses in the frame period, wherein the pulses arebunched together in a single area of the frame period; wherein thepulses are provided so that the pulses of a frame period has an averagepower of the frame period that is below a threshold average power, but apeak power of the pulses is above the threshold average power, whereinthe threshold is associated with a laser eye safety threshold; whereinall exposure times in a frame period are synchronized to a pulse duringthe frame period; and wherein the instructions cause the computingdevice to receive a signal from the sensors associated with thedetection of the projected light, and use the signal to perform depthmeasurement calculations, wherein the light is projected as ageometrical pattern.